Papers with heuristic method
Acceleration of Backpropagation in Linear Layers of Transformer Models Based on Gradient Structure (2026.eacl-srw)
Copied to clipboard
| Challenge: | a sparsity-exploiting backward pass is a memory-efficient way to accelerate LLM fine-tuning. |
| Approach: | They propose a method that exploits padding-induced gradient sparsity to accelerate backward computation. |
| Outcome: | The proposed method achieves a backward pass speedup of 2.15x on GLUE and 1.99x on reasoning benchmarks while maintaining memory usage identical to the regular PyTorch fine-tuning. |
On the Robustness of Question Rewriting Systems to Questions of Varying Hardness (2022.acl-long)
Copied to clipboard
| Challenge: | entailment : absence of questions classified based on their rewriting hardness or difficulty . enactment of QR system to rewrite context-dependent questions in CQA requires context knowledge . |
| Approach: | They propose a heuristic method to automatically classify questions into subsets of varying hardness . they then conduct a human evaluation to annotate the rewriting hardness of questions . |
| Outcome: | The proposed learning framework improves the overall performance compared to baselines. |
Adversarial Attacks on Knowledge Graph Embeddings via Instance Attribution Methods (2021.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge Graph Embeddings (KGE) are widely used for relational learning on large scale Knowledge . however, little is known about the security vulnerabilities that might disrupt their intended behaviour. |
| Approach: | They propose to use model-agnostic instance attribution methods to select adversarial deletions and a heuristic method to replace one of the two entities in each influential triple to generate adversarials. |
| Outcome: | The proposed methods outperform the state-of-the-art data poisoning attacks on KGE models and improve the MRR degradation by up to 62% over the baselines. |
Knowing-how & Knowing-that: A New Task for Machine Comprehension of User Manuals (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for machine reading comprehension of user manuals have trouble answering complex questions. |
| Approach: | They propose a knowing-how & knowing-that task that requires the model to answer factoid-style, procedure-style and inconsistent questions about user manuals. |
| Outcome: | The proposed model can answer factoid-style, procedure-style and inconsistent questions about user manuals. |
Transformers for Bridging Persian Dialects: Transliteration Model for Tajiki and Iranian Scripts (2024.lrec-main)
Copied to clipboard
| Challenge: | Despite its profound linguistic and cultural significance, Tajiki Persian remains a low-resource language with scant digitized datasets for computational applications. |
| Approach: | They propose to use Shahnameh, a seminal Persian epic poem, to train and assess Tajiki Persian transliteration models using two prominent sequence-to-sequence architectures: GRU with attention and transformer. |
| Outcome: | The proposed model outperforms pre-trained models with attention and transformer. |